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Abstract — This paper studies the stability of some recon- 
struction algorithms for compressed sensing in terms of the bit 
precision. Considering the fact that practical digital systems deal 
with discretized signals, we motivate the importance of the total 
number of accurate bits needed from the measurement outcomes 
in addition to the number of measurements. It is shown that if 
one uses a 2k x n Vandermonde matrix with roots on the unit 
circle as the measurement matrix, 0(£+k log j) bits of precision 
per measurement are sufficient to reconstruct a fc-sparse signal 
x £ R" with dynamic range (i.e., the absolute ratio between the 
largest and the smallest nonzero coefficients) at most 2 within £ 
bits of precision, hence identifying its correct support. Finally, we 
obtain an upper bound on the total number of required bits when 
the measurement matrix satisfies a restricted isometry property, 
which is in particular the case for random Fourier and Gaussian 
matrices. For very sparse signals, the upper bound on the number 
of required bits for Vandermonde matrices is shown to be better 
than this general upper bound. 

I. Introduction 

Compressed sensing is an emerging field which deals with 
new sampling techniques for sparse signals. The goal is to 
exploit the sparsity of the signal and try to reconstruct the 
signal using a number of linear measurements far below the 
signal's dimension. Formally, let x € M" be a /c-sparse vector 
(k <C n) and y = Ax, where A is an m x n measurement 
matrix with possibly complex entries, and y € C m is the 
observed vector. The main problem is to design a measurement 
matrix with m <C n for which there exists an efficient 
reconstruction algorithm that is able to reconstruct any k- 
sparse signal from the measurement y. It is easy to check 
that at least 2k measurements are required if all fc-sparse 
signals have to be distinguishable. However, it can be shown 
that k + 1 linear measurements with random coefficients are 
enough to reconstruct almost all signals almost surely [1], [2]. 
To reconstruct the signal we need to solve the following £o 
minimization problem 

(P ) min||i|| , Ax = y, 

which is known to be NP-hard for a generic matrix A. 

A key observation by Candes et al. [3], [4], [5] and 
Donoho [6] shows that if matrix A obeys a so-called restricted 
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isometric property (RlPfl which essentially requires that any 
set of up to k columns of A behaves close to an orthonormal 
system, then the signal can be exactly reconstructed using the 
following l\ minimization program 

(Pi) min||i||i, Ax = y. 

This is an easier problem compared to £ minimization 
and in particular can be solved in polynomial time using 
linear programming (LP) techniques. There are families of 
random matrices which satisfy the RIP with high probability 
if m, the number of rows, is large enough. Two examples of 
such families are given by random Gaussian measurements 
and random Fourier measurements. If A is a random matrix 
with i.i.d. Gaussian entries and m = 0(fclog?), or if A 
is constructed from m = 0(k\ogn) random rows of the 
n x n discrete Fourier transform matrix, then the matrix 
can be shown to satisfy the RIP with high probability [3]. 
Even though the results for random matrices hold with high 
probability, there is no known efficient way to verify if a 
random matrix satisfies the RIP. This motivates the problem 
of finding an explicit construction of a measurement matrix 
A with small number of measurements for which we can 
solve (P ) efficiently. It is shown that explicit matrices can 
be constructed based on group testing techniques [7] as well 
as expander graphs and randomness extractors [8], [9], [10], 
[11], [12]. 

A closer inspection reveals that (Pq) is an analog of the 
so-called syndrome decoding problem over real or complex 
numbers, and hence it is natural to expect that known tech- 
niques from coding theory might be applicable in compressed 
sensing. In particular, Akcakaya and Tarokh [13] show that 
several results known for the Reed-Solomon codes over finite 
fields can be extended to the field of complex numbers. 
Therefore, similar coding and decoding algorithms can be 
used for sensing sparse vectors over the real or complex field. 
Specifically, they show that it is possible to reconstruct any 
/c-sparse vector from only 2k measurements, which is the 
minimum number of measurements one can hope for, using 
0(n 2 ) arithmetic operations. 

All the above results hold under the assumption that mea- 
surements and arithmetic over real numbers are carried out 
precisely. However, in digital systems we generally cannot deal 

'This property was originally called uniform uncertainty principle (UUP) 
by Candes and Tao . 



with real numbers simply because we would need infinitely 
many bits to represent a real number. So it is inevitable 
to resort to truncated representations of real vectors. Thus 
a natural question to ask is how precise the measurement 
outcomes need to be so as to be able to reconstruct the original 
data within a target precision. In Section HU we will show 
how it becomes important to not only take the total number 
of measurements into account, but also the precision required 
from individual measurements. Together, these two quantities 
give a suitable measure of the amount of information (in bits) 
that needs to be extracted from the measurements in order to 
approximate the sparse signal. We will use a simple example 
to justify the point that if the precision of the measurements 
is allowed to be sufficiently high, even one measurement is 
sufficient to reconstruct discrete signals. 

The main result of this paper is a bit precision analysis of the 
syndrome decoding algorithm for Reed-Solomon codes when 
applied in the context of compressed sensing as a reconstruc- 
tion algorithm for Vandermonde measurement matrices [13]. 
The analysis is based on the assumption that the sparse signal 
is to be reconstructed within a certain chosen precision in 
the fixed-point model and the additional requirement that the 
support of the reconstructed vector is the same as that of the 
original signal. In particular, we show that if the dynamic range 
of x is at most 2 e then having each measurement available 
within 0(£ + k log bits of precision is sufficient to identify 
x within I bits, which is the minimum precision needed to 
ensure that the smallest nonzero entry of x is not confused 
with zero. Since we have a total of 2k measurements, the 
total number of bits required from the measurement outcomes 
is upper bounded by 0(£k + k 2 log §)• 

The rest of the paper is organized as follows. First, in Sec- 
tion|II]we motivate the total bit precision as a practical measure 
for assessing the quality of compressed sensing algorithms. 
Then, in Section [HI] we give a more rigorous definition of 
the problem that we consider and state our stability theorem 
for Vandermonde measurements and the syndrome decoding 
algorithm. Section [IV] gives the sketch of the proof for the 
stability theorem and in Section [V] we will upper bound the 
total number of bits required from the measurements obtained 
from matrices satisfying certain restricted isometry properties. 

II. The Importance of Bit Precision 

The main purpose of this work is to show that while 
the number of measurements is an important criterion for 
assessing the quality of a compressed sensing scheme, it is 
by itself insufficient without considering the precision needed 
for the reconstruction algorithm to work properly. Indeed, a 
more favorable approach than simply bounding the number 
of measurements would be to quantify the total amount of 
information (in bits) that needs to be extracted from the 
measurement outcomes so as to enable a reliable reconstruc- 
tion of the original signal within a pre-specified precision. 
Intuitively, a single real number can pack an infinite amount 
of information and for virtually all real world applications, 
either the signal to be measured is a priori known to be 



discrete (for example, the output of a sensor measuring the 
temperature over a long period of time), or is only needed 
within a certain pre-specified number of accurate bits. For 
all such cases, a single measurement is in principle capable 
to carry all the needed information. The following example 
illustrates this point. 

Example 1: Suppose that A is an m x n binary matrix that 
allows recovery of fc-sparse vectors over F2. Such a matrix 
can be obtained from a parity check matrix of a binary code 
with minimum distance at least k + 1. We now "compress" 
the matrix A into a vector a = (<ii, . . . , a n ) € K n such that 
any fc-sparse x € {0, 1}" can be exactly reconstructed from 
a ■ x G R. We define 

m 

ai :=5>C7\i)- 2*^+1)1), 

where A(j, i) denotes the the entry of A at the jth row and 
the ith column. This vector simply encodes all the rows of the 
matrix A by shifting each row by a sufficient amount to prevent 
any confusion, and a moment's thought reveals that indeed x 
can be uniquely reconstructed from a-x. However, by a simple 
counting, the number of rows of A has to be at least log (£) = 
il(k log(n/fc)), and we need at least m x |~log(fc + 1)] bits 
from a ■ x to be able to reconstruct x. Hence, although the 
number of measurements is extremely low, the total number 
of bits that we need to extract from the measurement has to 
be at least f2(fclogfclog(n/fc)). 

In general, a counting argument shows that matrices with 
entries from a small domain cannot be used to bring down 
the number of measurements below a certain level. This is 
captured in the proposition below: 

Proposition 1: Let A be an m X n matrix whose entries 
are integers in range [— 2 e ,2 e ]. Assume that A can be used 
for reconstruction of fc-sparse signals in K™. Then m = 

O ( k\og{n/k) \ 

Proof: The matrix A must be in particular able to distin- 
guish binary fc-sparse vectors. The number of such vectors is 
(?) . Let x be a fc-sparse binary vector and y :— Ax. Each entry 
of y must be an integer in range [— k2 e , k2 e ], and the number 
of vectors in K™ satisfying this property is (k2 i+1 + l) m , and 
this number must be lower bounded by the number of fc-sparse 
binary vectors. This gives the desired bound. ■ 
The above result explains why the entries of our single- 
measurement matrix had long binary representations. How- 
ever, as shown in [13], one can "break" this lower bound using 
Vandermonde matrices and achieve a total of 2k measure- 
ments. This special property of Vandermonde matrices is due 
to the fact that the entries of the matrix cannot be represented 
by bounded precision numbers and the amount of required 
precision must necessarily grow to infinity as n gets large. 
Hence, Vandermonde matrices use large precision in an essen- 
tial way and it becomes a crucial task to quantitatively analyze 

2 Using a similar construction, it is also easy to see that, allowing infinite 
precision in the measurements, it is information theoretically possible to 
uniquely identify any (not necessarily sparse) discrete vector x £ N™ using 
only one real linear measurement. 



the amount of precision that Vandermonde measurements need 
for making reliable reconstruction of sparse signals possible. 

III. Problem Definition and Main result 

We consider the problem of recovering a fc-sparse signal x G 
R n from m <C n discretized observations. Let A be an m x n 
matrix with possibly complex entries and y := y + e G C"\ 
be our observation vector, where y :— Ax and e G C m is the 
truncation noise in the observation. Throughout the paper we 
consider the fixed-point binary representation of real numbers 
and define precision as follows. 

Definition 1: We say that a vector z G C" is an approxima- 
tion of z G C™ within £ bits of precision (or £ accurate bits) 
if\\z-z\\ 00 /\\z\\ oc <2- e . 

Definition 2: The dynamic range of a nonzero vector x is 



defined as the ratio 



where i m „ and x„ 



are 

the largest and the smallest nonzero entries of x in absolute 
value, respectively. 

The problem is to find the sufficient precision for y such that 
we can ensure that a signal x with known dynamic range can 
be recovered with the correct support. This of course depends 
on the matrix A and the reconstruction algorithm. We pick as 
A a Vandermonde matrix with roots on the unit circle; namely, 

/ 1 1 ... 1 \ 
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where cij := exp (j 27r „ 1 )■ This choice of A is motivated 
by Reed-Solomon codes over finite fields. By the properties 
of Reed-Solomon codes if the "error pattern" (i.e., the vector 
x) is fc-sparse and we pick m = 2k then it can be uniquely 
identified from the measurement outcomes. As noted in [13] 
this property holds over the complex field as well and therefore 
if we use a Vandermonde matrix with distinct roots as the 
sensing matrix, we can exploit an analog of Reed-Solomon 
decoding algorithm over the complex field to reconstruct the 
signal. The Reed-Solomon decoding algorithm we use is the 
so-called syndrome decoding algorithm, where (for the case 
e = 0) the measurements yo, yi, . . . , y m -i are considered 
as syndromes from which we wish to find the corresponding 
error pattern (i.e., the vector x). The decoding algorithm is as 
follows. First, we solve the following Toeplitz linear system 



(1) 



for a nonzero solution, and let h(x) := ho + h±x + - ■ ■ + hkX k . 
From the theory of Reed-Solomon codes (cf. [14]) we know 
that h(x) is a multiple of the error locator polynomial L(x) := 
]^[ eeB (l — xa e ), where EC {1,2, ... ,n} is a set of size at 
most k containing the error positions (i.e., the support of x). 
Therefore the set of the zeros of h(x) determines a superset 
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of the error positions (and the exact set if h is a nonzero 
solution with the smallest degree). Having found a superset of 
error locations with size k, we can solve a k x k system of 
linear equations to find the actual error values. 

In the presence of truncation noise, it is natural to consider 
the same reconstruction method using the truncated syndrome 
vector y, and ask how stable the method is in this situation. 
The following theorem quantifies the amount of precision of 
the measurements needed to ensure correct recovery of the 
support: 

Theorem 1: If we use a 2k x n Vandermonde matrix with 
roots on the unit circle and observe the syndromes within 
0{£ + fclog^) accurate bits, then we can stably reconstruct 
any fc-sparse signal with dynamic range at most 2 . The 
reconstructed signal has the same support as the original 
signal and approximates its nonzero elements within I bits 
of precision. 

The proof of this theorem is sketched in Section [TV] 
Remark 1: For the fixed point model that we are consider- 
ing in this work, if the signal has dynamic range 2 , then we 
obviously need at least I bits of precision in the reconstructed 
signal to make sure that all the nonzero entries are being 
recovered with nonzero magnitudes. 

The theorem states that if we want to recover the correct 
support of a signal with dynamic range 2 e , 0(£ + fclog^) 
bits of precision per measurement is sufficient. Since we have 
2k measurements, the total number of measurement bits add 
up to 0(k£ + k 2 \ogj). In particular, if we consider binary 
signals for which £ = 0, then we will require 0(fc 2 log^-) 
bits in total. Nevertheless, this bound is k times larger than 
O(fclog^), the information theoretical lower bound on the 
number of required bits to identify fc-sparse binary signals. 

IV. Sketch of the Proof of Theorem [T] 

In this section we outline the proof of Theorem [Hand omit 
certain details due to space restrictions. In particular, we focus 
on the case where the support size of the sparse signal is 
exactly k or known to the decode^. The proof is done in three 
steps. First, we find sufficient precision for the hi so as to be 
able to detect the positions of nonzero elements, namely the 
roots of the error locator polynomial. Next, an upper bound is 
derived on the required precision for the yi so as to guarantee 
that the hi can be solved, with desired precision, from the set 
of linear equations given in ([]]). This upper bound depends on 
the condition number of a k x k matrix with entries given by 
the yi, for which an upper bound is derived in the last step. 

Step 1: In the presence of noise, ho, hi, . . . ,hk G C are 
noisy and h(x), assuming that ho = 1, is not necessarily equal 
to the error locator polynomial L(x). Thus, we need to make 
sure that the error in the hi is small enough to allow us to 

3 If the decoder knows the actual support size t, where t < k, it can discard 
all but 2t of the syndromes and reduce the problem to the case where the 
support size is exactly half the number of measurements. Otherwise, it can try 
various possibilities for t and find a list of up to k possible reconstructions 
that includes a correct approximation of x. It is not hard to see that only one 
of these reconstructions can reproduce the given syndromes within a sufficient 
precision, and thus, the decoder can always uniquely reconstruct x. 



reliably find the roots of L(x). From the choice of L(x) = 
Y\ eeE {^- — xa e ), the minimum nonzero magnitude of L(x) can 
be bounded as t 

\L{ x )\>k\ (He 

since each a e and the evaluation point x is an nth root of 
unity and the quantity 1 1 — xa e | is the length of a chorcQ on 
the unit circle whose corresponding angle is a distinct multiple 
of 27r/n (as the a e are distinct). 

Also, the magnitude of the error in evaluation of h(x) can 
be upper bounded by k 2 2~ lh if the hi are available within th 
bits of precision. This is because L(x) has k monomials with 
coefficients of magnitude at most k. 

Hence, to find the roots of L{x) correctly, it suffices to have 

fe 2 2 -4 <k i(^ ' 
\ n 

which can be satisfied by having the coefficient vector of h(x) 
within ih = 0{k log ?) bits of precision. 

Step 2: Now that we have a bound on the precision that we 
need for the h i7 we have to calculate the precision we need 
for y. We will use the following theorem. 

Theorem 2: [15, Theorem 7.2] Let Ax — b, where A is a 
square invertible matrix and x and b are vectors, and (A + 
AA)y = b + Ab, where ||AA|| < e\\E\\ and ||A6|| < e||/||. 
The matrix E and the vector / are arbitrary and || • || is any 
absolute norm. Furthermore, assume that e||A _1 || \\E\\ < 1. 
Then 
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as the norm function, 



\\x\\ 1 — e||^4— 1 
By picking E := A, f := b and 
we obtain the following corollary: 

Corollary 1: For a square matrix A let Ax = b and (A + 
AA)y = b + Ab, where HAA^ < e^Hoo and \\Ab\loo < 
e||6||oo, and assume that en 00 (A) < ~. Then 

^r„ y||o ° < 4e Koo (A), 



where (A) denotes the condition number of the matrix A 
with respect to the oo-norm. 

The corollary states that if we wish to obtain the solution 
x of a linear system Ax — b up to r accurate bits, i.e., \\x — 
x||oo/||a;||oo < 2~ r , it suffices to have ||AA||oo < e||A||oo and 
||A6||oo < e||6||oo with e = 0(2-(''+ lo s K ~( A ))). 

From Step 1, we know that we need O(klog^) bits of 
precision for the hi. Now suppose that the decoder receives a 
perturbed version of the syndrome vector and finds a nonzero 
solution for the (perturbed) system of linear equations ([T), 
namely, h := (ho, . . . , hk). We show that there is a solution 
h := (ho, . . . ,hk) for the original system (HJ that is suffi- 
ciently close to the perturbed solution, i.e., \\h— /i||oo/||/i||oo < 
2 -n(feiogf) ) as required by Step 1. 

4 Here for the sake of clarity we are neglecting the lower order term 
0(l/n 2 ) in the approximation on the length of the chord, but it should be 
clear that this will not affect the analysis. 



Denote by M the coefficient matrix of (HJ (prior to the 
perturbation of syndromes), and by M, the k x k minor of 
M obtained by removing the column corresponding to hi. 
Moreover, define y max as the largest syndrome in absolute 
value so that |j/ max | = ||y||oo, and note that each M, contains 
all the syndromes but one. It is always possible to set hi = hi, 
for some choice of i such that A/; contains an entry with 
magnitude | j/ max |, and we can rewrite (HJ as 



Mi(h k , 



, hi+i, hi-i 



,h ) T = 



-hi(y k -i,yk-i+i, ■ 



,V2k-i-l) 



(2) 



Here we mention a technicality that the precision of the 
yi is bounded relative to the largest coefficient ?/ max , which 
is not necessarily present on the right hand side of the 
above system. However, we can add an additional "dummy" 
equation y max hi = hiy max to the system and ensure that the 
requirements of Corollary[T]on the error bounds are fulfilled. It 
is easy to see that the new system will have condition number 
at most max{K OQ (Mi), k}. We can now apply Corollary Q] on 
the system given by (0 and find a sufficient precision for the 
yi. In particular, we conclude that 



0(k\og(n/k)) + log Koo (Mi) 



(3) 



bits of precision for the yi would be sufficient for finding the 
hi within the precision required by Step 1, and thus, the correct 
support of the sparse vector. 

Step 3: For the last step, we find a good upper bound on 
n OD (Mi). We know that ^^(Mi) < \fk ■ n(Mi), where k(-) 
denotes the condition number with respect to the li -norm. 
Thus we equivalently upper bound n(Mi). It is straightforward 
to see that Mi can be decomposed as Mj = DVkXkV k T , 
where Xk is a k x k diagonal matrix containing the nonzero 
coefficients of the sparse vector x on its diagonal, Vk is a fc x k 
Vandermonde matrix with roots on the unit circle, and D is 
a diagonal (and unitary) fc x k matrix containing appropriate 
powers of the <jj. Obviously, k(D) = 1. Moreover, as the 
dynamic range of x is bounded by 2 l , we have that n(Xk) = 
l^maxl/l^minl < 2^. Moreover, we use the following lemmas: 

Lemma 1: [16] The condition number of any complex k x k 
Vandermonde matrix V with roots on the unit circle is at most 
V2k. 

Lemma 2: If Q and R are square matrices with complex 
entries, then k(QR) < k(Q)k(R). 

The second lemma is easy to derive and we omit its 
proof. Altogether, we conclude that n(Mi) < k2 l+1 , which, 
combined with Step 2, implies that it suffices to have the yi 
within O (k log j + £) bits to correctly reconstruct the support 
of x. 

After we find the correct support, the reconstruction problem 
is reduced to a k x k system of linear equations defined 
by k columns of the Vandermonde measurement matrix and 
the corresponding measurement outcomes. As k columns of 
a Vandemonde matrix also form a Vandemonde matrix, by 
Lemma \T\ the condition number of the matrix defining the 
equations is at most \/2k. Hence, again using Corollary Q] 



knowing the measurement outcomes within 0(logk + I) bits 
would be sufficient for obtaining I bits of precision in the 
reconstruction of x. However, this number is less than the 
bound that we derived before for finding the correct support 
of x. This concludes the proof. 

V. A General Bound on Precision 
In the preceding section we obtained a bound on the amount 
of precision required for the measurements obtained from 
a Vandermonde measurement matrix for ensuring reliable 
recovery of the sparse signal using a particular reconstruction 
algorithm, namely, syndrome decoding. In this section, we 
consider a similar problem, but for a general class of measure- 
ment matrices satisfying a suitable restricted isometry property 
(RIP) and considering convex optimization as the recovery 
method. We remark that, as in the case of Vandermonde 
measurements, our main focus here is on the amount of 
information that needs to be extracted from the measurement 
outcomes, and we do not take the imprecision of numerical 
computations into account. In particular, we assume that the 
reconstruction algorithm uses an idealized computation model, 
but receives truncated measurement outcomes on its input. 

The bound that we obtain in this section is a direct corollary 
of a result by Candes et al. on robust recovery of sparse signals 
from inaccurate measurements [17]. We begin by recalling this 
result and the required notation. For a complex m x n matrix 
A (where m < n) and positive integer s, denote by the s- 
isometry constant S s the infimum over all choices of S that 
satisfy (l-£)||c||| < \\A'c\\\ < (1 + <5)||c||§, for every mxs 
submatrix A' of A and every c € C s . The following is the 
main result proved in [17]: 

Theorem 3: Suppose that the measurement matrix A satis- 
fies the restricted isometry property that 63s + 3<54 S < 2, for 
some positive integer s. Then for every e > 0, every s-sparse 
signal x S C", and y := Ax + e, ||e|| 2 < e, the solution x to 
the convex program 

(P 2 ) minllx'Hi, \\Ax' - y\\ 2 < e, 

satisfies \\x — x\\ < Ce, where C is a positive constant only 
depending on <54 S . 

As pointed out in [17], the program (P2) can be solved 
efficiently using known techniques from convex optimization. 
Now suppose in the sequel that A is an m x n matrix satisfying 
the isometry property needed above, and that we wish to 
reconstruct an approximation x of a fc-sparse signal x S C™ 
within at least I significant bits (in the fixed-point model) from 
y := Ax + e, where e is the rounding error, such that x has the 
same support as that of x. Obviously, for that to be possible 
the dynamic range of x must be at most 2 e , as otherwise a 
nonzero but small coefficient of x might be confused with 
zero. The following straightforward corollary of Theorem [3] 
quantifies the amount of precision needed for y: 

Corollary 2: Let e £ C m denote the quantization error in 
y. In order to ensure that ||x — ^Hoo/IMloo < 2 _f , it suffices 
to have 

Hellco , 2-< 
II2/H 00 Cky/m' 



The result states that if we have the measurements within 
Oil + logfc + logm) = 0(£ + logm) bits of precision, we 
can ensure that the program (P2) obtains a reconstruction that 
approximates x within I bits of precision, which in particular 
implies correct support recovery of x. Hence, the total number 
of bits needed from the measurements can be upper bounded 
by 0{m(l + logm)). 

As a concrete example, consider a measurement matrix A 
that outputs a set of m random Fourier coefficients of the 
signal. It is shown in [17] that, in order for A to satisfy the 
RIP needed by Corollary |2] with overwhelming probability, 
it is sufficient to take m = 0(k(logn) e ). Thus in this case, 
(9(fc(logn) 6 (£ + log/c + loglogn)) bits from the measurement 
vector y would be sufficient to reconstruct x within precision 
I. On the other hand, the upper bound that we obtained for 
the Vandermonde matrix with syndrome decoding is a total 
of 0{k£ + k 2 \og(n/k)) bits from y. The two bounds are 
incomparable, but the latter is better for very sparse signals 
(e.g., k = O(logn)). 
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